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1  .  INTRODUCTION 


Complex  problems  often  demand  complex  analytical  techniques.  So 
it  Is  with  attempts  to  understand  organizations:  reality,  defying 
more  prosaic  aids  to  understanding,  has  forced  us  to  develop  increasingly 
sophisticated  methods  of  observation,  description,  and  .analysis. 

Camlng  and  systems  simulation  are  two  such  methods.  Generally 
characterized  as  having  not  only  intuitive  appeal  —  bused  on  nice 
descriptive  properties  --‘but  also  a  solid  empirical  basis,  the  methods 
have  stimulated  the  development  of  a  large  and  expanding  professional 
community  and  literature.  What  the  community  does,  what  have  been  the 
trends  and  orders  of  magnitude  of  activity,  and  where  it  all  seems  to 
be  headed  are  questions  wortihy  of  reflection. 

It  in  amazing  that  after  all  these  years  there  is  still  such 
difficulty  in  sorting  out  what  we  mean  when  discussing  work  in  this 
area.  Definitional  "games"  often  degenerate  Into  fruitless  syntactic 


Any  views  expressed  in  this  paper  are  those  of  the  authors.  They 
should  not  be  interpreted  as  reflecting  the  views  of  The  Rand  Corporation 
or  the  official  opinion  or  policy  of  its  governmental  or  private  research 
sponsor;).  Papers  are  reproduced  by  The  Rand  Corporation  as  a  courtesy 
to  members  of  Its  staff. 

A  version  of  this  paper  was  originally  presented  at  a  Symposium  on 
Computer  Simulation  as  Related  to  Manpower  and  Personnel  Planning  at  the 
U.S.  Naval  Academy,  April  1971. 
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exercises;  however,  it  is  fairly  important  to  distinguish  four  separate 
areas  of  work:  analytic  models ,  machine  simulation ,  man-machine  games , 
and  free- form  games. 

Analytic  Models.  A  good  analytic  model  is  usually  quite  abstract, 

poor  in  number  of  variables  explicitly  considered,  but  rich  in  terms 

of  ease  of  manipulation  and  clarity  of  Insight.  Certain  forms  of 

warfare  have  been  characterized  quite  well  by  game  theoretic,  analytic 

models  in  which  two-sided  or  more  than  two-sided  combat  is  considered 
* 

explicitly.  For  example,  one  might  examine  the  optimal  behavior  of 
a  red  team  versus  a  blue  team  when  one  side's  strategy  is  not  fixed. 

In  a  situation  such  as  a  two-sided  war,  assuming  pure  opposition 
enables  one  to  consider  optimal  tactics,  deductions  of  an  opposition's 
best  strategy  may  lead  to  counter  maximization  strategies.  Unfortu¬ 
nately,  there  are  two  primary  reasons  why  it  is  generally  not  possible 
to  do  so.  Representations  of  combat  more  complex  than  simple  tactical 
encounters  are  usually  not  zero  sum,  i.e.,  it  is  usually  not  a  situa¬ 
tion  of  pure  opposition.  Elements  omitted  from  the  analysis  in  the 
interest  of  tractability  and  precision,  e.g.,  "human  factors,"  are 
often  crucial  to  understand  what  is  in  fact  going  on. 

Individuals  most  at  home  with  analytic  models  are  applied  mathe¬ 
maticians,  operations  research  analysts,  and  a  breed  of  creature  known 
as  the  "comptemiks,"  although  the  computer  is  frequently  used  purely 
as  an  analytic  aid. 

In  many  situations  the  aim  of  an  analytic  model  is  to  produce  a 
single  number  as  contrasted  with  multiple,  interrelated  indicators  of 
system  behavior.  There  may  be  more  than  a  single  number,  a  kill  prob¬ 
ability  or  a  specific  survival  level  for  instance,  that  is  of  analytic 
interest.  Frequently  a  single,  end  state  condition  for  a  system  is 
calculated.  This  contrasts  with  the  study  of  system  behavior  through 
time  where  end  states  may  not  be  of  interest  or  even  relevant. 

* 

Melvin  Dresher,  Games  of  Strategy:  Theory  and  Applications 
(Englewood  Cliffs:  Prentice  Hall,  1961),  for  example. 
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Machine  Simulation.  In  contrast  with  analytic  models,  machine 
simulations  very  frequently  involve  many  variables;  the  fetish  of 
"realism"  is  often  invoked  sanctimoniously.  Rationalizations  made 
on  behalf  of  the  computer  simulation  are  varied  and  nearly  as  numerous 
as  the  organizations  in  the  business  of  "modeling  anything,  anywhere, 
anytime"  —  for  a  consideration.  A  common  (and  frequently  valid) 
argument  is  that  mathematics  is  a  relatively  impoverished  descriptive 
language,  whereas  the  computer  allows  one  to  capture  the  richness  or 
robustness  of  a  real  system.  What  is  left  unstated  in  this  and  other 
pleas  is  the  enormous  price  one  is  likely  to  pay  to  approximate  that 
reality.  Simulations  and  the  "simulators"  responsible  for  their  design, 
implementation,  and  operation  are  much  harder  to  control  than  analytical 
models  or  modelers.  Not  only  are  there  fewer  discernible  scientific 
standards  available  to  aid  in  one's  evaluation  of  a  computer  simulation 
versus  a  mathematical  model,  there  is  little  or  no  general  agreement 
among  professionals  in  the  trade  as  to  what  standards  are  pertinent  and 
ought  to  be  developed. 

Certain  low  levels  of  misfeasance  and  abuse  are  tolerable  for  a 
profession;  as  the  stakes  increase,  however,  such  laxity  becomes  too 
costly  by  any  measure. 

Salesmanship  and  "heap  big  science"  may  well  be  the  undoing  of 
this  potentially  highly  useful  aspect  of  the  profession. 

Large-scale  computer  simulations  have  heretofore  been  rather 
easy  to  sell.  They  appear  reasonable;  furthermore,  they  provide  a 
wealth  of  material  for  a  three- or  four-star  "Dog  and  Pony  Show." 

Straight  lines,  curves  that  grow  exponentially,  curves  that  wiggle  up 
and  down,  smooth  curves,  discontinuous  curves  —  whatever  curve  you 
want  is  the  curve  you  get.  Whatever  system  you  want  defended  is  defen¬ 
sible  with  a  three-thousand  variable  black  box.  After  all,  who  questions 
"science"? 

The  main  types  of  people  in  the  machine  simulation  business  are 
analysts,  mathematicians,  systems  engineers,  "computemiks , "  and  occa¬ 
sionally  economists  and  defrocked  physical  scientists. 

Sometimes  machine  simulations  are  called  games.  Frequently  this 
is  meaningful;  often  it  is  not.  Man-machine  simulations  are  normally 
referred  to  as  games;  terminological  distinctions  quickly  tend  to  fuzz. 
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Man-machine  Simulation.  Man-machine  simulation  usually,  but  not 
always,  involves  a  digital  computer  with  some  roles  in  the  modeled 
system  being  played  by  people.  Two  distinctions  are  worth  making. 

People  may  be  used  merely  because  they  are  cheaper  than  the  software. 

Or,  people  may  be  used  because  human  factors  are  important  in  the 
analysis  of  the  situation.  An  example  will  serve  to  illustrate. 

Suppose  one  simulates  a  large,  but  relatively  straightforward,  logis¬ 
tics  system.  How  minor,  specific  roles  are  carried  out  is  of  little 
or  no  interest.  Given  programming  costs  and  this  low-level  interest, 
it  may  be  literally  cheaper  to  plug  a  captain  or  major  into  the  role 
for  a  two-  or  three-week  play  of  the  game  than  it  is  to  write  the 
software  to  simulate  him.  In  this  case  the  man  is  made  an  integral 
part  of  the  man-machine  program. 

In  other  situations,  for  other  analytic  purposes,  one  may  wish  to 
observe  how  the  man  behaves,  what  motivates  the  human  component,  and 
so  forth.  The  "man"  in  man-machine  is  then  clearly  performing  a  quite 
different  function. 

In  man-machine  gaming  and  simulation,  the  personnel  are  usually 
analysts,  systems  engineers,  experimental  psychologists,  social  psy¬ 
chologists,  and  economists.  Depending  upon  the  nature  and  purpose  of 
the  enterprise,  there  may  be  a  human  factors  emphasis  where  humans  are 
studied  and  not  merely  used  as  inputs.  With  a  systems  engineering 
emphasis,  humans  are  used  because  they  are  handy  and  relatively  cheap. 

In  the  latter  case,  the  orientation  is  more  toward  operations  research 
analysis  and  systems  engineering  than  it  is  to  experimental  psychology 
or  social  psychology. 

Distinctions  between  man-machine  and  free-form  games  (simulations) 
are  slight,  but  exist. 

Free-form  Gaming.  In  this  type  of  gaming  frequently  no  computer 
equipment  is  employed,  or  it  may  be  used  on  the  side  to  do  some  book¬ 
keeping.  Free-form  gaming  involves  individuals  operating  in  a  scenario. 
It  is  the  least  amenable  to  technical  control  of  the  four  types  of 
work  described. 
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Interesting  to  observe  is  that  free-form  gaming  has  the  least 
technical  control,  the  least  money  spent  on  it,  yet  it  receives  the 
most  publicity  and  is  done  at  the  highest  level.  In  the  Studies, 
Analysis  and  Gaming  Agency,  individuals  involved  in  some  of  the  games 
may  be  at  the  Secretary  level  and  at  the  three  and  four  star  Admiral 
or  General  level,  yet  the  amount  of  money  involved  may  be  of  the  order 
of  $10,000  per  game. 

Given  these  four  different  areas  of  study,  it  is  fairly  safe  to 
say  that  in  the  Defense  Department  only  two  of  them  are  characterized 
by  "big  money."  Man-machine  gaming  is  characterized  by  little  money 
relative  to  what  is  being  spent  on  simulation,  and  politico-military, 
free-form  gaming,  in  spite  of  the  way  it  appears,  is  characterized  by 
miniscule  money.  Cost  questions  are  important  and  shall  momentarily 
be  given  attention. 
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II.  DETAILING  THE  ART  FORMS  AND  THE  STATE  OF  THE  ART 

Besides  providing  the  interested  reader  with  several  concrete 
examples  of  each  of  the  simulation  or  game  types,  the  purposes,  use, 
and  limitations  of  each  may  be  summarized  to  help  assess  the  current 
state  of  the  art. 

More  on  analytic  models.  Good  analytic  models  help  to  spot  a 
"chicken  and  egg  problem"  that  once  recognized  can  usually  be  solved. 

The  point  bears  directly  on  the  relationship  among  rigorous  theoretical 
models,  empiricism,  and  data  gathering.  An  analytic  model  is  usually 
too  restricted  to  solve  an  actual  operational  problem  directly.  Because 
the  model  is  normally  clean  and  clear,  however,  it  can  provide  insight 
about  potential  difficulties,  indicate  where  additional  measurements 
are  most  needed,  and  identify  and  order  important  omissions. 

The  optimal  assignment  problem,  a  pure  analytic  model  created  by 

* 

John  von  Neumann,  is  a  case  in  point.  The  model  has  limited  interest 
for  purposes  of  direct,  operational  application.  Assuming  that  one 
knows  exactly  what  a  man  can  (and  will)  do,  constraining  analytic 
possibilities  with  linearity  assumptions  and  adopting  a  very  restricted 
view  of  personnel  assignment  reduce  direct  operational  utility.  Never¬ 
theless,  von  Neumann's  work  is  clean,  and  abstractly  captures  the 
core  of  an  important  problem. 

Analytic  work  on  the  Berlin  Airlift  provides  an  example  where 
initial  simple  analysis  yielded  to  more  complex  formulations  as  the 
problem  began  to  be  understood.  From  the  first  linear  program,  the 

John  von  Neumann,  "A  Certain  Two-person,  Zero-sum  Game  Equivalent 
to  the  Optimal  Assignment  Problem,"  in  H.  W.  Kohn  and  A.  W.  Tucker, 
eds.,  Annals  of  Mathematical  Study,  No.  28  (Princeton:  Princeton 
University  Press,  1953),  pp.  12-15. 

** 

A.  S.  Manne,  An  Application  of  Linear  Programming  to  Procure¬ 
ment  of  Transport  Aircraft,"  Management  Science,  Vol.  2,  No.  2 
(January  1956) . 
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analysis  evolved  to  dynamic  models  and  ended  up  as  a  combination  dynamic 
model  and  simulation.  The  evolutionary  process  was  probably,  in  a 
pragmatic  sense,  optimal.  One  might  conjecture  that  until  the  problem 
was  "learned"  with  simpler,  more  abstract  analyses,  realistic  repre¬ 
sentation  was  just  not  possible. 

To  review  the  last  thirty  years,  the  growth  in  the  use  and  power 
of  analytic  models  has  been  absolutely  astounding.  In  terms  of 
techniques,  new  insights,  and  amount  of  personnel,  growth  in  this 
period  has  probably  been  greater  than  in  the  100  preceding  years; 
quite  possibly  it  even  exceeds  all  previous  work.  In  our  opinion  the 
prospects  are  still  extremely  good  for  more  and  more  diversified  appli¬ 
cations  of  analytic  models  and  for  increased  use  of  the  computer  as 
an  analytical  aid:  not  as  a  simulator,  but  for  assistance  to  solve 
analytical  problems.  Furthermore,  phenomenal  advances  in  software 
and  input/output  aids  have  occurred  in  the  last  ten  years.  The  outlook 
is  bright  indeed  for  serious  scholarship,  respectable  problem  solving, 
and  careful  application.  The  status  and  prospects  for  our  three  other 
types  are  less  certain,  although  we  tend  to  be  optimistic. 

And  on  to  machine  simulations .  There  are  literally  hundreds  of 
machine  simulations  and  models  in  the  Department  of  Defense's  current, 
active  inventory.  Force  structure  and  weapons  system  evaluation  models 
are  representative  of  much  of  the  activity,  although  a  more  precise 
description  of  the  field  is  forthcoming. 

We  are  certain,  however,  that  machine  simulation  has  been  over¬ 
sold  in  the  last  ten  years,  and  the  shakedown  is  now  taking  place. 

The  process  is  healthy,  albeit  painful  in  certain  quarters.  Much  has 
been  learned  that  contributes  to  the  beginnings  of  a  serious  professionali¬ 
zation  of  work  in  simulation.  Problems  have  at  least  been  delineated. 

Three  major  problem  areas  that  must  still  be  viewed  in  considerable 
detail  are  (a)  modeling  and  specification,  (b)  data  gathering  and 
validation,  and  (c)  sensitivity  analysis  and  question  formulation. 

Each  of  these  is  a  lengthy  subject  in  itself.  However,  their  importance 
may  be  summarized  simply.  There  is  no  such  thing  as  a  good  general 
purpose  simulation. 
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Anyone  who  tells  you  that  he  has  a  good  general  purpose  simulation 
falls  into  one  of  two  categories:  he  is  an  unmitigated  charlatan 
trying  to  sell  a  bill  of  goods;  or,  as  is  frequently  •‘he  case,  he  is 
a  rather  good  specialist  who  has  been  oversold  by  his  own  specialization 
to  the  point  where  he  believes  a  good  special  purpose  simulator  to  be 
a  general  purpose  simulator.  Examples  of  this  can  be  seen  in  the 
construction  of  simulation  languages.  Three  come  to  mind.  There  is 
Professor  Jay  Forrester's  language  Dynamo,  best  described  before  it 
was  modernized  as,  "If  the  world  were  an  RLC  circuit,  Forrester  would 
be  King."  Languages  like  GPSS  or  Simscript  are  general-purpose  lan¬ 
guages  only  if  the  world  happens  to  be  a  production  inventory  scheduling 
problem.  Understandably,  such  is  not  the  case. 

The  warning  to  beware  of  general-purpose  simulations  is  not  meant 
to  depreciate  the  desire  to  have  something  that  solves  all  problems. 

It  deals  instead  with  the  difficulties  of  modeling,  specification,  data 
gathering,  validation,  sensitivity  analysis,  and  question  formulation. 
These  are  all  nasty,  boring,  hard,  dirty,  tough  subjects  that  make  the 
difference  between  a  viable,  useful  simulation  and  something  that  is 
only  useful  to  supply  nicely  colored,  wiggling  graphs  for  stellar 
"Dog  and  Pony  Shows." 

The  man  in  the  machine  (as  subject  and  object).  We  could,  if  we 
wished,  immediately  apply  an  operation  test  by  regarding  how  man- 
machine  gaming  is  doing  as  an  indication  of  how  it  should  be  doing. 
Waxing  lyrical  and  biblical,  the  phrase  that  comes  to  mind  is,  "And 
they  shall  beat  their  gaming  laboratories  into  lunch  rooms."  (Experi¬ 
ence  in  the  aerospace  industry  tells  the  tale,  especially  in  the  last 
five  years.)  Such  was  not  always  the  state  of  affairs  and  one  need 
only  recall  earlier,  happier  times  and  highly  respectable  activities 
to  make  the  point. 

The  Logistics  Simulation  Laboratory  at  The  Rand  Corporation  was 
one  example  of  a  man-machine  simulation  in  which  individuals  were  used 
more  as  an  integral  part  of  the  machinery  than  as  subjects  for  human 
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factor  analyses.  The  team  performance  study  initiated  by  Professor 
John  Kennedy  at  Rand  became  the  founding  work  for  the  System  Develop¬ 
ment  Corporation  and  is  characterized  predominantly  as  human  factors 
** 

studies . 

Man-machine  gaming  and  simulation  was,  at  its  peak  around  5-7  years 
ago,  oversold.  Currently,  we  suspect  that  it  is  being  undersold.  The 
key  need  in  man-machine  gaming  is  to  build  up  a  basic  inventory  that 
stresses  substantive  content  in  various  areas  of  interest. 


"Free- form"  doesn't  have  to  mean  "loose  and  sloppy."  A  prime 
example  of  free-form  gaming  that  should  be  most  familiar  is  PME  — 
Political  Military  Exercises  done  at  and  for  the  Joint  Chiefs  of  Staff 
by  SAGA  and  others.  These  games  are  aimed  at  the  strategic  level 
and  involve  relatively  high-priced  personnel.  They  highlight  some  of 
the  difficulties  in  costing.  When  we  try  to  cost  a  game  or  a  simu¬ 
lation,  we  have  to  decide  not  only  how  to  cost  the  physical  resources 
but  what  sort  of  value  we  wish  to  impute  to  the  time  of  the  people  who 
play  the  game.  If  we  only  account  for  physical  resources,  all  that  a 
top  level  political  military  exercise  needs  are  a  couple  of  rooms,  a 
few  people  who  are  already  assigned  to  the  Pentagon,  some  pencils, 
some  pads  of  paper,  and  a  bit  of  TV  tape.  The  costs  actually  depend 
on  how  we  evaluate  the  worth  of  the  time  of  the  top  echelon  teams. 

There  has  been  an  extremely  small  amount  of  money  spent  on  free¬ 
form  gaming.  With  respect  to  the  four  types  of  gaming  and  simulation, 
it  appears  that  publicity  is  inversely  related  to  the  amount  expended 
in  each  area  —  we  suspect  inversely  as  something  like  the  fourth 


M.  A.  Geioler  and  A.  S.  Ginsberg,  Man-Machine  Simulation  Experience 
(Santa  Monica:  The  Rand  Corporation,  P-321A,  August  1965). 

** 

R.  L.  Chapman  and  J.  L.  Kennedy,  The  Background  and  Implications 
of  the  Systems  Research  Laboratory  Studies  (Santa  Monica:  The  Rand 
Corporation,  P-740,  1955).  See  also,  N.  Frederickson,  "Factors  in 
In-Basket  Performance,"  Psychological  Monographs ,  No.  22  (1962),  for 
another  good  example  of  the  genre. 

•kirk 

For  a  basic,  clear  introduction  to  this  type  of  game,  see 
Herbert  Goldhamer  and  Hans  Speier,  "Some  Observations  on  Political 
Gaming,"  World  Politics ,  Vol.  12  (1959),  pp.  71-83. 
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power.  Free-form  gaming  has  a  few  good  practitioners;  it  attracts 
the  most  publicity;  it  is  very  hard  to  measure  the  product;  and  it  is 
extremely  difficult  to  describe  whether  the  art  form  has  improved  in 
the  last  few  years.  We  suspect  that,  as  it  cost  a  minimal  amount 
initially,  a  little  more  money  and  some  careful  work  in  this  area  would 
probably  be  well  worth  the  effort. 


1.1. 


III.  OK  VALUE  AND  PRICE 


There  is  an  old  rule  in  simulation  worth  indicating:  A  simulation 
oosts  almost  anything  that  the  bookkeepers  want  to  make  it  cost.  The 
reasons  for  this  are  complicated  and  merit  lengthy  treatment.  Generally, 
the  matter  of  cost  depends  heavily  on  whether  one  gets  information 
from  ongoing  processes  or  whether  special  work  to  gather  the  data  must 
be  instituted.  The  joint  costs  of  data  gathering  can  be  assigned  in 
virtually  any  way.  A  standard  operations  research  plea  that  forcing 
one  to  look  at  data  saves  untold  millions  for  the  "relatively"  trifling 
costs  of  data  acquisition  ignores  an  important  bureaucratic  cost:  people 
do  not  like  to  have  their  organizations  and  procedures  deranged  merely 
to  collect  data. 

The  question  of  value  is  intimately  related  first  to  how  a  simu¬ 
lation  or  game  is  used  and  then  to  what  it  costs.  Three  aspects  are 
of  particular  interest:  the  level  and  type  of  use,  the  measure  of  worth 
of  use,  and  the  cost. 

Levels  of  use.  Four  distinct  levels  of  use  may  be  noted. 

Strategic,  top  decisionmaking  usage  is  exemplified  by  the  political- 
military  exercise.  Even  though  it  is  an  open  question  as  to  how  use¬ 
ful  it  has  been,  PME  is  explicitly  labeled  for  use  at  the  top  strategic 
level,  and  it  interfaces  directly  with  the  people  who  could  be  described 
as  participants  at  that  level.  Weapons  system  procurement  studies 
and  simulations  are  implicitly  designed  for  use  at  the  top  level.  It 
appears  that  to  some  extent  they  are  used  for  advocacy  purposes  to 
bolster  one  side  of  the  budget  process  and  not  the  other.  After  a 
large-scale  weapons  evaluation  systems  simulation  has  been  performed, 
it  appears  that  the  information  is  used  primarily,  if  not  exclusively, 
by  a  given  general  or  admiral  rather  than  by  other  interested  civilian 
participants.  We  do  not  assert  that  this  is  necessarily  good,  or  bad. 

It  is  merely  what  appears  to  be  the  process. 

The  next  level  of  interest  is  at  the  operational/bureaucratic  and 
the  scientific  interface.  Most  studies,  simulations,  and  much  human 
factors  work  fall  into  this  category.  Here  we  see  contractors  and 


subcontractors  working  closely  with  their  opposite  numbers  in  the  DoD 
bureaucracy.  The  problems  are  meant  to  be  operational  and  relatively 
specific. 

A  third  level  is  the  scientific  and  advisory  interface  with  opera¬ 
tions.  This  describes  a  fair  amount  of  the  work  sponsored  by  the 
Advanced  Research  Projects  Agency.  Investigations,  such  as  some  human 
factors  studies,  are  typically  of  the  sort  where  there  is  little 
interest  in  immediate  application,  but  much  in  the  building  up  of  a 
basic  supply  of  knowledge. 

The  fourth  level  is  the  more  purely  basic  scientific  and  experi¬ 
mental  work  that  utilizes  top  professionals  in  a  scientific  advisory 
capacity.  When  one  uses  a  scientific  advisor,  expectations  are  that 
if  not  actively  engaged  in  producing  basic  scientific  knowledge,  the 
advisor  is  at  least  actively  aware  of  the  developments  in  that  body 
of  knowledge.  Included  at  this  level  is  work  in  basic  experimental 
psychology,  programming,  game  theory,  hardware  and  laboratory  develop¬ 
ment,  and  human  factors  studies. 

What  is  use  Worth ?  What  are  some  appropriate  measures  for  level 
and  type  of  use?  There  is  nothing  approaching  a  consensus  on  these 
matters;  worse,  there  is  not  even  much  discussion  about  them. 

One  standard  applied  in  some  areas  is  the  market  measure  —  will 
it  sell,  does  it  get  constantly  funded?  This  is  a  relatively  unsatis¬ 
factory  criterion.  It  is  a  first-order  measure,  but  the  type  that 
allows  us  to  ask  somewhat  more  profound  questions. 

Other  possible  measures  might  be:  "How  many  top  level  briefings 
are  given,  based  on  this  particular  study?"  Or  one  might  use  an 
index  such  as,  "How  many  times  is  this  particular  study  referred  to  in 
making  operational  decisions?"  None  of  these  are  perfect  in  and  of 
themselves,  but  it  is  at  least  useful  to  keep  them  all  in  mind. 

Another  way  of  constructing  a  measure  would  be  to  gather  interview 
data  on  specific  decisions.  Was  the  decision  to  build  a  piece  of 
hardware  based  upon  the  use  of  a  certain  study?  And  if  it  was,  by  whom, 
and  what  form  of  rationalization  was  given  for  the  way  in  which  the 
study  was  employed? 
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In  some  cases,  the  number  of  scientific  publications  might  be  a 
measure.  And  last,  but  by  no  means  least,  a  key  measure  is:  Who 
instigated  the  study?  This  question  has  to  be  viewed  along  two  levels 
at  least.  Those  who  pay  the  bill  are  not  necessarily  those  who  requested 
the  work.  It  is  essential  to  ask  fundamental  questions  about  initiation 
if  we  are  to  begin  understanding  the  capabilities  of  a  model  or  simu¬ 
lation.  Did  Admiral  "X",  for  example,  really  want  a  particular  study, 
or  did  it  happen  that  there  were  unused  and  forfeitable  dollars  remain¬ 
ing  at  the  end  of  the  fiscal  year?  To  avoid  the  unpardonable,  in  the 
latter  instances,  the  ever-present  Ph.D.  with  a  pet  idea  gets  employed. 

He  does  "his"  study;  he  has  a  good  time;  and  the  study  is  immediately 
shelved  and  quickly  forgotten. 

It  is  essential  to  know  details  of  initiation  so  as  to  be  able 
to  trace  any  game  or  simulation  through  its  "life,"  from  initiation 
to  promotion  and  implementation,  and  then  on  through  evaluation  and 
termination.  Who  wants  it?  Who  sells  it  on  what  grounds,  for  what 
purposes?  Who  conceives  and  actually  builds  it?  Who  appraises  it? 

Who  decides  to  renew  or  retire  it?  This  procession  through  the 
decisionmaking  system  is  doubtless  complex;  nonetheless,  it  is  important 
to  be  aware  of  the  process,  and  understanding  initiation  is  important. 

Costs.  Costs  are  enormously  hard  to  determine.  It  is  difficult 
to  obtain  a  relevant,  simple  cost  description.  To  some  extent,  random 
and  somewhat  irrelevant  accounting  figures  are  easy  to  obtain;  but  as 
the  work  we  are  discussing  involves  a  quantity  of  invisible  costs, 
overhead  costs,  jointly  shared  facilities  and  jointly  used  products, 
a  meaningful  costing  procedure  poses  a  deep  scientific  problem  that  is 
nowhere  near  being  satisfactorily  solved. 

Though  we  are  well  aware  of  these  and  other  difficulties,  we  can 
give  basic,  brief  guesses  about  costs  for  each  type  of  work. 

Straight  analysis  and  analytic  models  are  doubtless  relatively 
inexpensive.  Realistically  so,  for  the  only  overhead  support  required 
for  much  of  a  competent  professional's  enterprise  is  a  pencil  and  pad 
of  paper,  or  maybe  chalk  and  a  board.  The  IBM  Executive  typewriter, 
carbon  ribbons,  and  high-grade  bond  paner  most  likely  have  hindered 
developments  in  this  area  by  the  imputation  of  God-given,  immutable 
properties  to. tentative  formulations.  Frequently  what  one  needs  are 
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quick  and  dirty  analytical  models  that  do  not  look  nice  but  do  contain 
good  concepts.  The  medium  is  not  the  message;  models  are  the  message. 

To  prevent  defilement  by  infidels  and  to  encourage  thought,  self- 
destruction  of  both  model  and  medium  is  recommended. 

Simulation,  as  presently  done,  is  usually  an  extremely  expensive 
pastime  whose  value  may  be  inversely  related  to  the  cube  of  the  fourth 
power  of  the  output.  The  cost  is  also  probably  related  as  the  cube  of 
the  fourth  power  of  the  number  of  parameters  and  variables  dreamed  up 
for  use  in  the  model.  Simulation  is  generally  expensive  and  the 
validity  problems  are  difficult;  therefore  one  does  not  like  to  risk 
making  many  general  observations  except  to  note  that  it  helps  to  know 
one's  business. 

Man-machine  simulation  has  more  or  less  the  same  spotty  record. 

It  too  is  expensive,  but  both  with  simulation  and  man-machine  simulation 
there  are  examples  in  which,  although  the  work  was  expensive,  it  was 
distinctly  worth  the  effort.  One  or  two  reasonably  well-defined  and 
understood  questions,  asked  early,  the  answers  to  which  were  important, 
seemed  to  be  a  common  property  of  successful  examples.  Not  only  was 
the  answer  important,  the  principals  were  intelligent  enough  to 
recognize  the  answer  when  confronted  with  it.  Frequently  in  a  large- 
scale  simulation  an  answer  may  appear,  but  it  passes  unnoticed  because 
no  one  recognizes  it.  After  deciding  what  the  question  is,  it  pays  to 
ask  what  a  reasonable  answer  might  look  like.  Neither  step  is  often 
taken,  as  attested  by  the  warehouses  that  are  full  of  unanalyzed  computer 
print-out,  and  boxes  of  punched  cards. 

Costs  for  free-form  gaming  are  relatively  low,  although  the 
publicity  is  disproportionately  large.  Given  both  absolute  and  relative 
low  levels  of  present  expenditures,  a  few  additional  pennies  spent  on 
this  type  of  game  would  certainly  do  no  harm  —  it  might  even  stimulate 
more  serious  scholarship. 

How  big  are  the  sums  involved?  Our  guess,  and  these  are  pure 
guesses  informed  by  an  exhaustive  reading  in  the  area  but  not  by  access 
to  much  inside  information,  is  that  free-form  gaming  accounts  for  less 
than  $.1  million  per  year.  All  other  games,  simulations,  and  studies 
account  for  anywhere  between  $150  and  $300  million  per  year,  depending 
upon  how  one  defines  his  terms  and  totals  his  accounts. 
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IV.  PROSPECTS  FOR  GROWTH 


The  profession  and  the  field  have  grown  like  Topsy.  Professionali¬ 
zation  has  finally  begun.  Coordination,  standards,  communication, 
independent  appraisal,  review,  and  accounting  activities  are  all  healthy 
indicators  of  a  new  awareness  that  can  only  be  in  the  profession's  own 
collective  and  individual  self-interest.  Let  us  stress,  however,  that 
the  real  work  has  only  just  begun. 

Review  boards.  With  respect  to  the  use  of  professional  review 
boards  take  this  example. 

Probably  the  most  elaborate  gaming  facility  in  the  world  is  at 
the  Naval  War  College  at  Newport,  Rhode  Island.  The  Navy  Electronic 
Warfare  Simulator,  NEWS,  is  perhaps  the  biggest  electronic  tube  device 
machine  ever  constructed  by  man.  It  cost,  in  terms  of  1950  dollars, 
around  $12  million.  This  particular  facility  has  long  been  run  without, 
to  the  best  of  our  knowledge,  any  continuous,  professional,  non-Navy 
review  board.  Such  a  board  —  if  it  existed  —  quite  naturally  would 
not  only  assess  current  operations  and  procedures,  but  would  be  expected 
to  introject  knowledge  concerning  advances  in  simulation,  game  theory, 
and  analytical  methods,  and  even  general  advances  in  war  gaming.  By 
definition,  what  we  mean  by  continuous  and  professional  excludes  the 
practice  of  taking  in  one  another's  wash. 

Communication,  documentation  and  use.  It  is  our  opinion  that 
it  is  a  dangerous  policy  to  keep  the  level  of  professional  intercon¬ 
nection  so  low.  The  needs  for  better  coordination,  documentation,  and 
studies  of  use  specifically  at  the  operational,  experimental,  and 
administrative  interfaces  are  great.  It  is  not  enough  that  a  study  be 
finished  according  to  contract  specifications;  what  becomes  of  the 
study  and  how  the  study  gets  used  are  far  more  important  considerations. 
Learning,  kinds  and  amounts,  must  be  evaluated  so  that  a  rational 
expenditure  of  resources  can  begin.  Perhaps  expenditures  to  ensure 
that  evaluations  are  done  on  how  previous  studies  were  used  are  more 
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important  than  running  three  or  four  extra  studies.  Weapons  evaluation 
studies,  for  instance,  that  are  either  unused  or  misused  may  be  worse 
than  no  studies  at  all.  We  strongly  suspect  that  ill-conceived  pro¬ 
cedures  of  stewardship  (e.g.,  military  activities  with  high  rates  of 
turnover  lack  personnel  continuity,  producing  short  institutional 
memories),  coupled  with  highly  uneven  documentation  standards  and 
procedures  may  account  for  low  and/or  ineffective  model  usage.  If 
no  one  remembers  why  an  existing  model  was  built,  for  whom,  and  what 
its  peculiar  operating  characteristics  are,  there  may  be  a  tendency  to 
either  use  it  incorrectly  or  build  a  new  model  from  scratch.  If  there 
is  little  or  no  documentation,  this  potentially  wasteful  tendency 
becomes  a  practical  certainty. 

General  purpose  simulations.  There  is  no  such  thing  as  a  general 
purpose  applied  simulation;  beware  of  anyone  who  promises  it. 

Either  knotf  or  "get  the  business. "  There  is  no  such  thing  as  a 
simulation  substitute  for  knowing  one's  business.  If  the  business  is 
assigning  personnel,  the  individual  who  assumes  that  function  ought 
to  know  his  business  thoroughly  before  he  contracts  to  build  a  large 
simulator  that  will  give  all  the  answers.  One  of  the  real  dangers  is 
the  poverty  of  the  scientific  interface  between  those  who  know  their 
business  and  those  who  try  to  turn  their  models  into  a  business. 

Human  factors  and  basic  knowledge.  A  fifth  and  final  point  is 
that  there  is  a  desperate  need  for  large-scale  basic  knowledge  of 
human  factors.  There  is  an  extreme  danger,  given  funding  patterns 
at  the  present  moment,  that  no  one  has  estimated  the  shortage  of  supply 
of  basic  knowledge.  There  is  great  and  legitimate  need  for  basic 
work  on  topics  such  as  studies  on  panic  behavior,  studies  in  threat 
and  confrontation,  and  even  basic  studies  on  human  motivation.  The 
list  is  large.  Desperately  needed,  also,  is  a  review  of  the  kinds  of 
basic  knowledge  that  need  expanding,  how  these  tasks  should  be  inter¬ 
faced  with  the  operational  establishment,  and  what  form  of  funding 
might  be  brought  to  bear. 
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Render  unto  the  National  Science  Foundation  that  which  is  NSF's. 
However,  let  us  begin  to  make  very  sure  that  the  interface,  the  connection 
between  NSF  and  other  sources  of  research  funds,  is  what  it  needs  to 
be. 


18 


V .  TO  RECAP 


The  future  for  gaming  and  simulation  is  quite  good.  There  is  a 
steady  building  up  of  demand.  There  has  been  an  enormous  improvement 
in  technology  and  analysis.  Far  more  personnel  are  now  available 
than  ten  years  ago.  Control  ir.  determining  how  effective  the  growth 
of  this  area  is  going  to  be  is  in  each  of  our  hands.  If  we  fail  to 
exercise  that  control,  it  is  going  to  be  easy  to  butcher  another  couple 
of  hundred  million  dollars.  If  we  fail  to  build  up  a  basic  knowledge 
store,  we  are  going  to  retard  drastically  needed  improvements  in  the 
system  for  another  five  to  ten  years  or  more.  If  we  fail  to  integrate 
top  decisionmaking  systems  and  the  bureaucratic  interface  with  research 
work  by  failing  to  provide  decent  scientific  advice  for  the  operators, 
we  may  produce  work  that  superficially  can  be  judged  as  good,  but  in 
actuality  will  by  no  means  be  cost  effective. 

Finally,  the  future  is  good;  the  demand  is  growing;  knowledge  and 
technology  are  improving;  but  the  growth  has  to  be  controlled  and 
the  control  is  primarily  up  to  us. 


